Software testing is an integral, costly and time-consuming activity in the software development life cycle. Each time an application is built it must be intensively tested. A typical software testing cycle comprises a development team writing code which implements features of a specification and building the features into an executable program. A test team examines the specification and determines which features of the specification have been implemented and writes tests to test those features. The test team runs the tests against the executable program and reports any errors. The reported errors are corrected and the test team runs more tests and reports any further errors found. Once no further errors have been found the development team may choose to implement further features from the specification or change the specification and implement those features or release the executable program to users. If more features are implemented or if features are changed the test software must be updated to account for this.
This cycle will typically be repeated thousands of times in the lifetime of a program.
FIG. 1 illustrates a typical test system wherein a test application 100 generates 120 commands for executing a function at a command line interface 125 of an application under test 105. Software which tests command line interfaces is normally required to ensure that all parts of the output 110 of each command are within an expected range of values are specified by the specification. During the development of a project the specification will often change. This requires that the test application 100 is changed at the same time, for example, testing new functionality which would produce new data output 115. This is very time consuming and error prone.
The application under test 105 in response to a command being executed at its command line interface 125 produces an output 110 in the form of tables (also known as actual data) with a name at the top of each column as shown in example 1 below.